Hands and face tracking for VR applications

In this paper, we present a robust real-time 3D tracking system of human hands and face. This system can be used as a perceptual interface for virtual reality activities in a workbench environment. The main advantage of our system is that the human, that is in front of the virtual reality device, do not needs any type of marker or special suite. The system includes a real time color segmentation module to detect in real-time the skin-color pixels present in the images. The results of this skin-color segmentation are skin-color blobs that are the inputs of a data association module that labels the blobs pixels using a set of object state hypothesis from previous frames. The 2D tracking results are used for the 3D reconstruction of hands and face in order to obtain the 3D positions of these limbs. Finally, we present several results using the H-ANIM standard to show the system output performance.